LEGO-Net: Learning Regular Rearrangements of Objects in Rooms

Humans universally dislike the task of cleaning up a messy room. If machines were to help us with this task, they must understand human criteria for regular arrangements, such as several types of symmetry, co-linearity or co-circularity, spacing uniformity in linear or circular patterns, and further inter-object relationships that relate to style and functionality. Previous approaches for this task relied on human input to explicitly specify goal state, or synthesized scenes from scratch -- but such methods do not address the rearrangement of existing messy scenes without providing a goal state. In this paper, we present LEGO-Net, a data-driven transformer-based iterative method for learning regular rearrangement of objects in messy rooms. LEGO-Net is partly inspired by diffusion models -- it starts with an initial messy state and iteratively "de-noises'' the position and orientation of objects to a regular state while reducing the distance traveled. Given randomly perturbed object positions and orientations in an existing dataset of professionally-arranged scenes, our method is trained to recover a regular re-arrangement. Results demonstrate that our method is able to reliably rearrange room scenes and outperform other methods. We additionally propose a metric for evaluating regularity in room arrangements using number-theoretic machinery.Comment: Project page: https://ivl.cs.brown.edu/projects/lego-ne

Exploring Empathy and a Range of Emotions Towards Protest Photographs

Images are a powerful medium known to induce empathy and emotional response in people. In political protests it has the power for a people-initiated policy change and signifies the deep symbolism of a political system. In this study, we aim to quantify the range of emotional connection a person experiences for photographs of a farmers' protest.The protest was the headlines in all media at the time this experiment was conducted and had polarized public opinion. Each photograph is identified to have a set of physical and semantic features. The three selected features were presence of police, gender and close-up (vs.long-shot) in the frame. The intensity on a range of emotions (fear, disgust, anger, sadness, optimism, pessimism, surprise, shock, happiness, and respect) experienced by the viewer for each feature was collected. By statistical and dimensionality analyses, we isolate and identify influencing factors in an image. We found that the presence of police in aggressive actions and close-up shots of had the highest variation in the emotional responses of participants. Interestingly, the gender of the protesters did not show statistically significant effects. The findings from the exploratory investigation highlights the powerful role photographic features have on emotional responses of people, an understudied but critical factor in a world immersed in social media

Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields

Coordinate-based implicit neural networks, or neural fields, have emerged as useful representations of shape and appearance in 3D computer vision. Despite advances, however, it remains challenging to build neural fields for categories of objects without datasets like ShapeNet that provide "canonicalized" object instances that are consistently aligned for their 3D position and orientation (pose). We present Canonical Field Network (CaFi-Net), a self-supervised method to canonicalize the 3D pose of instances from an object category represented as neural fields, specifically neural radiance fields (NeRFs). CaFi-Net directly learns from continuous and noisy radiance fields using a Siamese network architecture that is designed to extract equivariant field features for category-level canonicalization. During inference, our method takes pre-trained neural radiance fields of novel object instances at arbitrary 3D pose and estimates a canonical field with consistent 3D pose across the entire category. Extensive experiments on a new dataset of 1300 NeRF models across 13 object categories show that our method matches or exceeds the performance of 3D point cloud-based methods

Semantic code search via equational reasoning

© 2020 Owner/Author. We present a new approach to semantic code search based on equational reasoning, and the Yogo tool implementing this approach. Our approach works by considering not only the dataflow graph of a function, but also the dataflow graphs of all equivalent functions reachable via a set of rewrite rules. In doing so, it can recognize an operation even if it uses alternate APIs, is in a different but mathematically-equivalent form, is split apart with temporary variables, or is interleaved with other code. Furthermore, it can recognize when code is an instance of some higher-level concept such as iterating through a file. Because of this, from a single query, Yogo can find equivalent code in multiple languages. Our evaluation further shows the utility of Yogo beyond code search: encoding a buggy pattern as a Yogo query, we found a bug in Oracle's Graal compiler which had been missed by a hand-written static analyzer designed for that exact kind of bug. Yogo is built on the Cubix multi-language infrastructure, and currently supports Java and Python